Applicant: LIPTONetal. 
Appl.No. 10/667,148 

In the Claims: 

The following listing of claims will replace all prior versions and/or listings of claims in 
the application: 
Listing of Claims: 

1 . (Currently Amended) A method of implementing an intelligent video surveillance 
system, comprising: 

obtaining a frame sequence from an input video stream; 

executing a first-pass method for each frame of the frame sequence, the first-pass method 
comprising the steps of: 

aligning the frame with a scene model; and 
updating a background statistical model; and 
finalizing the background statistical model; 

executing a second-pass method for each frame of the frame sequence, the second-pass 
method comprising the steps of: 

labeling each region of the frame; and 

performing spatial/temporal filtering of the regions of the frame; 
identifying and classifying objects using the labeled and filtered regions; and 
analyzing behaviors of at least one of the objects. 

2. (Original) A computer-readable medium comprising software implementing the method of 
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3. (Original) An intelligent video surveillance system comprising a computer system 
comprising: 

a computer; and 

a computer-readable medium according to Claim 2. 

4. (Original) A method of implementing an intelligent video surveillance system, comprising: 

obtaining a frame sequence from a video stream; 

for each frame in the frame sequence, performing the following steps: 

aligning the frame with a scene model; 

building a background statistical model; 

labeling the regions of the frame; and 

performing spatial/temporal filtering; 
identifying and classifying objects based on the results of the labeling and filtering; and 
analyzing behaviors of at least one object. 

5. (Original) A computer-readable medium comprising software implementing the method of 
Claim 4. 

6. (Original) An intelligent video surveillance system comprising a computer system 
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comprising: 

a computer; and 

a computer-readable medium according to Claim 5. 

7. (Original) A method of implementing an intelligent video surveillance system, comprising: 

obtaining a frame sequence from a video stream; 
for each frame in the frame sequence, performing the following steps: 
aligning the frame with a scene model; 

building a background statistical model and a secondary statistical model; 

labeling the regions of the frame; and 

performing spatial/temporal filtering; 
identifying and classifying objects based on the results of the labeling and filtering; and 
analyzing behaviors of at least one object. 

8. (Original) A computer-readable medium comprising software implementing the method of 
Claim 7. 

9. (Original) An intelligent video surveillance system comprising a computer system 
comprising: 

a computer; and 
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a computer-readable medium according to Claim 8. 

10. (Original) A method of implementing an intelligent video surveillance system, comprising: 

segmenting video into foreground and background components, the segmenting 
comprising: 

obtaining a sequence of video frames; 

building and updating at least one background statistical model for each region of 
the video frames, based on the video frames; and 

assigning labels to the regions, based on the at least one background statistical 

model; 

identifying and classifying objects based on the labeled regions; and 
analyzing behaviors of at least one object. 

1 1 . (Original) A computer-readable medium comprising software implementing the method of 
Claim 10. 

12. (Original) An intelligent video surveillance system comprising a computer system 
comprising: 

a computer; and 

a computer-readable medium according to Claim 11. 
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13. (Previously Presented) The method of Claim 1, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking at least one of the objects. 

14. (Previously Presented) The method of Claim 1, further comprising: 

creating at least one rule to detect at least one specific activity; 

wherein said analyzing behaviors of at least one of the objects includes applying the at 
least one rule. 

15. (Previously Presented) The method of Claim 14, wherein said at least one rule includes at 
least one virtual tripwire and determining when the at least one virtual tripwire is crossed. 

16. (Previously Presented) The method of Claim 14, wherein said at least one rule includes a 
definition of at least one area and the determining at least one of when an object enters, when an 
object leaves, and when "an object loiters in the at least one area. 

17. (Previously Presented) The method of Claim 14, wherein said at least one rule includes at 
least one of determining when an object is added to a scene and determining when an object is 
removed from a scene. 
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18. (Previously Presented) A method of implementing an automated closed-circuit television 
(CCTV) surveillance system, comprising: 

providing CCTV equipment generating an input video stream; and 
implementing the method of Claim 1. 

19. (Previously Presented) A method of implementing an automated security system, 
comprising the method of Claim 1. 

20. (Previously Presented) A method of implementing an automated anti-terrorism system, 
comprising the method of Claim 1. 

21 . (Previously Presented) A method of implementing an automated market research system, 
comprising the method of Claim 1 . 

22. (Previously Presented) The method of Claim 21, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking behaviors of at least one of the objects in at least one retail location. 
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23. (Previously Presented) A method of implementing an automated traffic monitoring 
system, comprising the method of Claim 1. 

24. (Previously Presented) The method of Claim 23, wherein said analyzing behaviors of at 
least one of the objects comprises at least one of: 

detecting wrong- way traffic; 
detecting a broken-down vehicle; 
detecting an accident; and 
detecting a road blockage. 

25. (Previously Presented) A method of implementing a video compression system 
comprising the method of Claim 1 . 

26. (Previously Presented) The method of Claim 4, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking at least one of the objects. 

27. (Previously Presented) The method of Claim 4, further comprising: 

creating at least one rule to detect at least one specific activity; 
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wherein said analyzing behaviors of at least one of the objects includes applying the at 
least one rule. 

28. (Previously Presented) The method of Claim 27, wherein said at least one rule includes at 
least one virtual tripwire and determining when the at least one virtual tripwire is crossed. 

29. (Previously Presented) The method of Claim 27, wherein said at least one rule includes a 
definition of at least one area and the determining at least one of when an object enters, when an 
object leaves, and when an object loiters in the at least one area. 

30. (Previously Presented) The method of Claim 27, wherein said at least one rule includes at 
least one of determining when an object is added to a scene and determining when an object is 
removed from a scene. 

3 1 . (Previously Presented) A method of implementing an automated closed-circuit television 
(CCTV) surveillance system, comprising: 

providing CCTV equipment generating an input video stream; and 
implementing the method of Claim 4. 



-9- 



Applicant: LIPTON et al. 
Appl.No. 10/667,148 

32. (Previously Presented) A method of implementing an automated security system, 
comprising the method of Claim 4. 

33. (Previously Presented) A method of implementing an automated anti-terrorism system, 
comprising the method of Claim 4. 

34. (Previously Presented) A method of implementing an automated market research system, 
comprising the method of Claim 4. 

35. (Previously Presented) The method of Claim 34, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking behaviors of at least one of the objects in at least one retail location. 

36. (Previously Presented) A method of implementing an automated traffic monitoring 
system, comprising the method of Claim 4. 

37. (Previously Presented) The method of Claim 36, wherein said analyzing behaviors of at 
least one of the objects comprises at least one of: 

detecting wrong- way traffic; 
detecting a broken-down vehicle; 
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detecting an accident; and 
detecting a road blockage. 

38. (Previously Presented) A method of implementing a video compression system, 
comprising the method of Claim 4. 

39. (Previously Presented) The method of Claim 7, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking at least one of the objects. 

40. (Previously Presented) The method of Claim 7, further comprising: 

creating at least one rule to detect at least one specific activity; 

wherein said analyzing behaviors of at least one of the objects includes applying the at 
least one rule. 



41 . (Previously Presented) The method of Claim 40, wherein said at least one rule includes at 
least one virtual tripwire and determining when the at least one virtual tripwire is crossed. 
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42. (Previously Presented) The method of Claim 40, wherein said at least one rule includes a 
definition of at least one area and the determining at least one of when an object enters, when an 
object leaves, and when an object loiters in the at least one area. 

43. (Previously Presented) The method of Claim 40, wherein said at least one rule includes at 
least one of determining when an object is added to a scene and determining when an object is 
removed from a scene. 

44. (Previously Presented) A method of implementing an automated closed-circuit television 
(CCTV) surveillance system, comprising: 

providing CCTV equipment generating an input video stream; and 
implementing the method of Claim 7. 

45. (Previously Presented) A method of implementing an automated security system, 
comprising the method of Claim 7. 

46. (Previously Presented) A method of implementing an automated anti-terrorism system, 
comprising the method of Claim 7. 
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47. (Previously Presented) A method of implementing an automated market research system, 
comprising the method of Claim 7. 

48. (Previously Presented) The method of Claim 47, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking behaviors of at least one of the objects in at least one retail location. 

49. (Previously Presented) A method of implementing an automated traffic monitoring 
system, comprising the method of Claim 7. 

50. (Previously Presented) The method of Claim 49, wherein said analyzing behaviors of at 
least one of the objects comprises at least one of: 

detecting wrong-way traffic; 
detecting a broken-down vehicle; 
detecting an accident; and 
detecting a road blockage. 

5 1 . (Previously Presented) A method of implementing a video compression system, 
comprising the method of Claim 7. 
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52. (Previously Presented) The method of Claim 10, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking at least one of the objects. 

53. (Previously Presented) The method of Claim 10, further comprising: 

creating at least one rule to detect at least one specific activity; 
wherein said analyzing behaviors of at least one of the objects includes applying the at 
least one rule. 

54. (Previously Presented) The method of Claim 53, wherein said at least one rule includes at 
least one virtual tripwire and determining when the at least one virtual tripwire is crossed. 

55. (Previously Presented) The method of Claim 53, wherein said at least one rule includes a 
definition of at least one area and the determining at least one of when an object enters, when an 
object leaves, and when an object loiters in the at least one area. 

56. (Previously Presented) The method of Claim 53, wherein said at least one rule includes at 
least one of determining when an object is added to a scene and determining when an object is 
removed from a scene. 
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57. (Previously Presented) A method of implementing an automated closed-circuit television 
(CCTV) surveillance system, comprising: 

providing CCTV equipment generating an input video stream; and 
implementing the method of Claim 10. 

58. (Previously Presented) A method of implementing an automated security system, 
comprising the method of Claim 10. 

59. (Previously Presented) A method of implementing an automated anti-terrorism system, 
comprising the method of Claim 10. 

60. (Previously Presented) A method of implementing an automated market research system, 
comprising the method of Claim 10. 

6 1 . (Previously Presented) The method of Claim 60, wherein said analyzing behaviors of at 
least one of the objects comprises: 

tracking behaviors of at least one of the objects in at least one retail location. 

62. (Previously Presented) A method of implementing an automated traffic monitoring 
system, comprising the method of Claim 10. 
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63. (Previously Presented) The method of Claim 62, wherein said analyzing behaviors of at 
least one of the objects comprises at least one of: 

detecting wrong-way traffic; 
detecting a broken-down vehicle; 
detecting an accident; and 
detecting a road blockage. 

64. (Previously Presented) A method of implementing a video compression system, 
comprising the method of Claim 4. 

65. (New) An apparatus for intelligent video surveillance adapted to perform the method 
comprising: 

obtaining a frame sequence from an input video stream; 

executing a first-pass method for each frame of the frame sequence, the first-pass method 
comprising the steps of: 

aligning the frame with a scene model; and 
updating a background statistical model; 
finalizing the background statistical model; 

executing a second-pass method for each frame of the frame sequence, the second-pass 
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method comprising the steps of: 

labeling each region of the frame; and 

performing spatial/temporal filtering of the regions of the frame; 
identifying and classifying objects using the labeled and filtered regions; and 
analyzing behaviors of at least one of the objects. 



66. (New) The apparatus of claim 65 wherein the apparatus comprises application-specific 
hardware to emulate a computer and/or software adapted to perform said obtaining, said 
executing a first-path method, said finalizing, said executing a second-path method, said 
identifying, and said analyzing. 

67. (New) An apparatus for intelligent video surveillance adapted to perform the method 
comprising: 

obtaining a frame sequence from a video stream; 

for each frame in the frame sequence, performing the following steps: 

aligning the frame with a scene model; 

building a background statistical model; 

labeling the regions of the frame; and 

performing spatial/temporal filtering; 
identifying and classifying objects based on the results of the labeling and filtering; and 
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analyzing behaviors of at least one object. 

68. (New) The apparatus of claim 67 wherein the apparatus comprises application-specific 
hardware to emulate a computer and/or software adapted to perform said obtaining, said aligning, 
said building, said labeling, said filtering, said identifying, and said analyzing. 

69. (New) An apparatus for intelligent video surveillance adapted to perform the method 
comprising: 

segmenting video into foreground and background components, the segmenting 
comprising: 

obtaining a sequence of video frames; 

building and updating at least one background statistical model for each region of 
the video frames, based on the video frames; and 

assigning labels to the regions, based on the at least one background statistical 

model; 

identifying and classifying objects based on the labeled regions; and 
analyzing behaviors of at least one object. 
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70. (New) The apparatus of claim 69 wherein the apparatus comprises application-specific 
hardware to emulate a computer and/or software adapted to perform said segmenting, said 
obtaining, said building, said assigning, said identifying, and said analyzing. 
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